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Background / Context: 

Life is powered by the basic processes of biological energy transfer. These processes 
include: 1) photosynthesis (PS) by which “food” and oxygen for most living things is made and 
2) cellular respiration (CR) by which some energy is transferred from them to ATP. ATP is the 
chemical energy currency used by all living things to power the activities we call “life.” 

Decades of research have documented student confusion regarding biological energy 
transfer, which has been identified as a cross-cutting concept and educational priority in multiple 
policy documents (NRC, 2012; NGSS, 2013; DOE, 2014). Recently, Andy Anderson’s group 
reported that only 10% of high school students became committed after instruction to tracing 
matter and energy in carbon-transforming processes (Jin and Anderson, 2012). However, we fear 
a new ‘tracing tool’ developed to remedy this situation (Dauer et al., 2014) will create difficulties 
when students attempt to transfer their understanding of energy to other disciplines. As specified 
by the Next Generation Science Standards for PS and CR, this tool scaffolds declarative 
knowledge of conservation of matter and energy with no rationale for why or how this occurs in 
living things. In addition, the tool’s use of twist ties on ‘bonds’ of carbon compounds to indicate 
‘energy’ appears to support multiple misconceptions including: energy is separate from the 
interactions of matter (Quinn, 2014), energy exists alone in a fuel, and that energy is found in 
bonds. That last idea makes it difficult for chemistry students to learn that energy must be added 
to break all bonds (Cooper and Klymkowsky, 2013). Our biological energy transfer pedagogy 
also uses models, but it provides a rationale for both why and how this occurs, which should 
encourage the belief that life depends upon physical laws (Barak, 1997, 1999). Here we report 
significant long-term effects of this approach upon student learning and briefly introduce how it 
is uniquely positioned to support a coherent presentation of energy across disciplines. 

Purpose / Objective / Research Question / Focus of Study: 

The SUN approach to biological energy transfer education is fundamentally different 
from past practices that trace chemical and energy inputs and outputs. The SUN approach uses a 
hydrogen fuel cell to convince learners that electrons can move from one substance to another 
based on differential attraction. With a hydrogen fuel cell, learners can see a propeller turning 
when only hydrogen and oxygen gases are present. The movement of electrons from the 
hydrogen fuel to the oxygen acceptor is supported with an animation of the invisible electron 
transfer and resultant products made. Living things run on microscopic fuel cells called 
mitochondria where the electron donor is food and the ultimate acceptor is oxygen. The SUN 
Project scaffolds understanding of this process with physical and digital manipulatives. Students 
configure nested trays with moveable components so as to enact the salient processes that 
transfer matter and energy in the mitochondrion. Energy is harvested not in the turning of 
propellers, but in the eventual rotation of a nanomachine that produces a chemical essential for 
life called ATP. SUN scaffolds include a mechanically manipulated replica with supporting 
animations. Food making in chloroplasts depends upon light to initiate electron movement. The 
materials show that many of the intervening processes mimic those in the mitochondrion. 

The purpose of this cluster, randomized controlled trial (RCT) was to determine the long- 
term efficacy of the SUN Project intervention on student understanding regarding why and how 
CR and PS occur. Students participated in regular biology classes within schools randomly 
assigned to either the treatment (T) or control (C) condition. At the same time the achievement of 
a small group of AP biology students who did not use the SUN materials provided another 
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meaningful context for interpreting outcomes. The research questions (RQ) of this project asked 
during the first year were: 

• RQ1. Does the SUN Project significantly impact long-term student learning? 

• RQ2. Does the SUN Project significantly impact student confidence in their 
understanding? 

• RQS. What variables moderate student outcomes? 

• RQ4. How much do SUN vs. Control students value their instructional materials? 

• RQ5. How do the outcomes of treatment students compare to that of a small group of AP 
biology students who are not using the SUN materials? 

Setting: 

This cluster RCT was conducted for one year in primarily Wisconsin high schools. 

About half of the schools were rural while -30% of the schools had <80% white population. The 
AP group also included rural/non-rural and ethnically varied schools. Only one teacher 
participated from each school with two exceptions: there were two similar “urban” schools that 
each contained two participating teachers with one school assigned to each group (T and C). 
Population / Participants / Subjects: 

Approximately 24% of assigned teachers declined participation before the study began. 
(Please see “Results” for evidence of initial T and C equivalence nonetheless.) Once begun, the 
subjects were 779 high school students in one regular biology class of each of 38 participating 
teachers. Because one control class was excluded for a pretest administration error, 19 treatment 
and 18 control teachers with 770 students ultimately participated. Schools were matched by high 
school freshman class size and ethnic makeup before random assignment. Because all except two 
schools fielded one regular biology teacher, randomization was essentially by teacher. To 
minimize bias toward high achievers, teachers designated which class would participate prior to 
the start of the school year. Seven AP biology teachers with 115 students provided another type 
of comparison. While not part of the RCT, they provided some meaning to any gains achieved. 
Intervention / Program / Practice: 

Treatment group teachers attended the summer SUN workshop for two weeks during 
which they significantly improved both their knowledge and self-efficacy with very large effect 
sizes, even one year later (Batiza et al, 2013). During the workshop, the teachers became familiar 
with these materials and revised their curricula for the coming year, incorporating only what they 
found valuable. All teachers deposited implementation data online every two months and were 
supported with regional meetings twice a year. In the classes of treatment and control group 
teachers as well as in the classes of a small group of AP biology teachers, tests (that teachers 
were asked not to examine) were administered at the beginning and end of the year. Project staff 
visited each classroom in order to collect student evaluations of teaching materials used and to 
provide support. Control teachers and AP biology teachers attended the workshop at the end of 
the year as compensation. Data was collected for another year when all were using the SUN 
materials, but this paper deals only with the first year, which constituted the cluster, RCT. 
Research Design: 

During the first year, a cluster, RCT was conducted regarding the effectiveness of the 
SUN curriculum vs. ‘business as usual controls.” The small AP group served as an additional 
comparison group to help determine the meaning of outcomes. The pre/post assessments 
included both a multiple choice (MC) assessment on CR and PS and a drawing with explanation 
assessment regarding CR. Table 1 shows the MC post assessment core understandings mapped to 
the Bloom’s Taxonomy level of the 20 items (Please insert Table 1). The 7 core concepts 
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addressed CR and PS and their comparison to each other and to a hydrogen fuel cell. Item 
analysis indicated high validity and difficulty; 35% of items required higher level thinking skills. 

Data Collection and Analysis: 

Pre and post student data was collected unexamined by the teacher and mailed to the PI. 
Data was coded into SPSS files by undergraduate and graduate assistants overseen by the lead 
evaluator. HLM analysis was conducted by HLM an expert with SAS PROC MIXED. 

Findings / Results: (please insert Table 2 here) 

Descriptive statistics. Based on the 770 participating regular biology and 1 15 AP students 
(excluding the 9 non-participants as above), missing variables varied between 3% and 13%. See 
Table 2 for the computed means and standard deviations of the variables broken down by 
treatment. Table 2 shows the treatment and control groups were initially equivalent in terms of 
previous GPA, multiple choice (MCI) and drawing (DR1) content assessments, confidence in 
their answers (CNF1), and gender distribution (Please insert Table 2). Similarly, treatment and 
control group teachers had equivalent prior knowledge and self-efficacy (Batiza et al., 2013). 
Correlations among the variables of analysis are presented in Table 3. (Please insert Table 3.) 

RQ1 & 2: Does the SUN Project significantly impact student learning about CR and PS, 
and confidence in responses? Because students were nested within teachers, hierarchical linear 
models (HLM) were used to test the treatment effect (Model 1, Table 4). (Please insert Table 4.) 
The dependent variables of the HLM analyses were (1) students’ post multiple choice scores 
based on 20 items, (2) students’ post drawing scores, and (3) students’ post confidence scores. 
The control variables were the pre-scores*, GPA, gender, and days elapsed between the end of 
instruction* and the assessment. Because the data were not missing completely at random, we 
used Multiple Imputation to deal with the missing data (Enders, 2010). 

The results in Table 5 show that students in the SUN group achieved significantly higher 
than Controls on post multiple choice scores ( y 0l =1.31, p<.001, effect size = 0.55), drawing 
scores ( y 01 = 1.19, p< .001, effect size =0.70), and confidence ( y 0l = 7.37, p< .001, effect size 

=0.62), controlling for pre-test scores, GPA, gender, and days elapsed (See Table 5 for all the 
other parameter estimates in the model). (Please insert Table 5) In general, GPA had a positive 
relationship with the outcome variables and days elapsed had a negative relationship with the 
outcome variables. Pre-test scores had a significant positive effect only on the post-test scores of 
multiple choice and confidence. Males had significantly higher scores than females on the 
multiple choice assessment and more confidence in their responses. 

RQ3: Is the impact of SUN Project moderated by students’ GPA and gender? Model 2 in 
Table 4 was used to test these interaction effects. Only GPA significantly moderated the 
treatment effect on student drawing and multiple choice scores. The treatment effect was greater 
for students with higher GPA. More specifically, as GPA increased by 1 point, the treatment 
effect increased by 0.61 on multiple choice and 0.86 on drawing. 

RQ4: Do SUN students value their instructional materials more than controls? Students 
were asked to rate their learning materials in terms of effectiveness on a 0 to 4 point scale. A 
score of ‘4’ represented ‘extremely useful’ perceived effectiveness. Individual students’ ratings 
were aggregated at the class level. A one-way ANOVA was conducted to test if there were 
significant differences in class mean values among SUN, Control, and AP classes. The mean 


The pre-scores for multiple choice questions are based on 15 items. 

* End of instruction was defined as end of unit instruction. Teachers were asked not to review before posttest 
administration at the end of the year. 
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ratings for SUN, Control, and AP classes were 2.36 (n=19, SD=0.30), 2.11 (n=18, SD=0.26), and 
2.41 (n=7, SD=0.33), respectively. There was a significant difference among the means 
[F(2,41)=4.21, p=0.0221. The Tukey post hoc pairwise comparisons showed that SUN students 
rated their materials significantly higher than did Controls by 0.26 points (p=0.037). 

RQ5 Comparison of SUN and Control vs. non-SUN AP. To better interpret the treatment 
effect, we further compared SUN and Control with AP students (See Tables 2 and 6) (Please 
insert Table 6). Controlling for gender, GPA, pre-test scores, and days elapsed, SUN was not 
significantly different from AP on the MC exam (p=.06) or confidence in responses (p=.44). 
However, SUN was 1.01 points higher than AP on drawing on average (pc. 05). Conversely, the 
Control group was significantly lower than AP by 1.47 points on the MC exam (pc. 05), but not 
significantly different from AP on the drawing assessment (p=.44) or in confidence (p=.33). 

For the drawing assessment, students were asked, “ Draw what occurs in the 
mitochondrion during CELLULAR RESPIRATION. Show what happens because of moving 
electrons. Label your drawing. Explain your drawing in the box below. ” (For a Treatment 
response please see Figure 1.) (Please insert Figure 1.) Each drawing was graded by three 
graders with a 19-item rubric of >.90 inter-rater reliability. When examining the percent correct, 
the difference between the SUN and Control responses is striking (Figure 2) as is the similarity 
of SUN and AP (Figure 3). (Please insert Figures 2 and 3.) The end product of CR (ATP, item 
19) was indicated by only 10% of controls, while 28% of SUN and 41% of the AP group did so. 
Other items best achieved by the SUN group (Criteria 4, 5 and 10, Figure 2), reflect salient 
structures in the mitochondrion (pumps and the synthase) and the path of electrons that map to 
the manipulatives used. As found earlier by Jin and Anderson (2012), none of the groups tracked 
matter well in terms of reactants and products (Criteria 6 and 7, Figures 2 and 3). 

Conclusions: 

These results of a cluster, RCT demonstrate that the SUN intervention allows regular 
biology students to significantly increase their long-term (>2.5 months after instruction on 
average) conceptual understanding of CR and PS relative to Controls with a moderate effect size 
of 0.55 on the MC exam and a large effect size of .70 on a drawing assessment. The SUN 
approach is value added since SUN students either achieved like AP biology students or 
surpassed them (See Table 6) when significant variables were controlled. Notably the SUN 
students valued their curricular materials more than Controls. However no group tracked matter 
well (See Criteria 6 and 7 in Figures 2 and 3). There is a long history of the inability of both 
teachers and students to learn about biological energy transfer (discussed in Batiza et al., 2013). 
The SUN approach has broken this pattern for teachers (Batiza et al., 2013) and here shows 
moderate and large long-term effects on student learning. It is interesting that the SUN approach 
allows students to think about, discuss, and demonstrate mechanism using physical models. 
Pedagogies that largely depend upon teacher or ‘textbook correctness’ (Russ et al., 2008) include 
the failed strategies largely those used by the controls during this study. Although we did not 
initiate this study with a concern for the treatment of energy across disciplines, it appears that the 
SUN approach is uniquely positioned to allow for a mechanistic treatment. The SUN tools 
evoke simplistic but fundamental interactions such as ‘like charges repel and opposites attract’ to 
help explain why these processes occur. Similar interactions drive some energy transfer 
processes in chemistry and physics. The fact that fundamental interactions drive energy transfer 
is one of four concepts that make it evident that the meaning of energy is equivalent across 
disciplines. Therefore this presentation will include reflections on how these unifying concepts 
could guide development of coherent energy curricula across HS biology, chemistry and physics. 
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Appendix B. Tables and Figures 


Table 1. Percent of items within the 20-item SUN Project Student 
Biological Energy Survey Related to 7 Concept Categories. 

Bloom’s Taxonomy Level is Indicated. 

(Consensus of 5 key personnel, all of whom are or have been teachers. 

Some items span more than one category) 

Percent of 

Measure 

Concepts 

KNOWLEDGE 

COMPREHENSION 

APPLICATION 

ANALYSIS/ 

SYNTHESIS/ 

EVALUATION 

12% 

Fundamental laws of 
matter and energy 


6.25% 

2.5% 

3.33% 

31% 

Cellular respiration (CR) 

10% 

11.25% 


10% 

27% 

Photosynthesis (PS) 

10% 

12.5% 

2.5% 

1.67% 

13% 

Comparison of CR and PS 




13.33% 

5% 

Comparison of CR to 
hydrogen fuel cell 


5% 



7% 

Related aspects of 
biology 



5% 

1.67% 

5% 

How ATP powers life 




5% 

100% 

TOTALS 

20% 

35% 

10% 

35% 


Table 2. Means and standard deviations of variables in the SUN, Control, and AP in Year 1 



SUN 

Control 

AP 

Student level variables 

Mean 

(N=399) 

SD 

Mean 

(N=371) 

SD 

Mean 

(N=115) 

SD 

MATE 

.46 

» 

.49 

» 

.35 

- 

GPA 

2.92 

.85 

3.08 

.74 

3.59 

.48 

MCI 

4.88 

1.94 

4.98 

1.80 

5.94 

2.17 

MC2 

7.84 

2.83 

6.64 

2.31 

9.16 

3.37 

DR1 

.03 

.19 

.03 

.17 

.49 

1.02 

DR2 

2.19 

3.04 

.21 

.63 

2.03 

3.18 

CNF1 

2.11 

.76 

2.06 

.66 

2.44 

.70 

CNF2 

2.96 

.72 

2.59 

.71 

2.96 

.89 








Teacher/class level variables 

Mean 

(N=19) 

SD 

Mean 

(N=18) 

SD 

Mean 

(N=8) 

SD 

DAYS Elapsed 

84.13 

50.05 

85.94 

62.34 

107.88 

74.00 
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Table 3. Correlations among variables in Year 1 



CNFl 

CNF2 

MCI 

MC2 

GPA 

MALE 

SUN 

Control 

Days 

elapsed 

DR1 

DR2 

CNFl 

1 











CNF2 

.438" 

1 










MCI 

.137" 

.136" 

1 









MC2 

.151" 

.355" 

.295" 

1 








GPA 

-.139” 

.045 

.194" 

.244" 

1 







MALE 

.195" 

.163" 

.045 

.081* 

-.189" 

1 






SUN 

-.027 

.186" 

-.083' 

.109" 

-.163” 

-.001 

1 





Control 

-.080* 

-.241" 

-.032 

-.263” 

.015 

.060 

-.770” 

1 




Days_elapsed 

.073" 

-.092* 

.034 

-.010" 

.062 

.072* 

-.058 

-.007 

1 



DR1 

.204" 

.233" 

.207" 

.316" 

.144" 

.058 

-.123” 

-.115** 

.133" 

1 


DR2 

.038 

.369" 

.121" 

.450" 

.237” 

-.018 

.305" 

-.378" 

-.140" 

.287" 

1 


Notes. MCl=pretest multiple choice, MC2=posttest multiple choice, CNFl=pretest confidence, CNF2=posttest 
confidence, SUN=SUN treatment, DAYS=days elapsed between instruction and test, DRl=pretest drawing, 
DR2=posttest drawing; * correlation is significant at the 0.05 level; ** correlation is significant at the 0.01 level. 


Table 4. HLM Models 


Model 1 

Level 1: Y tj = fi 0j + fd x Pr <?.. + P 2 GPA ij + 0 3 Gender v + e tj 

Level 2: fi 0j = y 00 + y 0X SUN . + y 02 Days _elapsed j+U 0j 

(Note. For drawing, the slope of GPA was random rather than fixed) 

Model 2 

Level 1: Y tj = 0 Oj + f5 Xj Pr e (j + p 2j GPA ij + /3 3j Gender - + e tj 

Level 2: J3 0j = y 00 + y 0l SUN ; + y 02 Days _ elapsed ; + U 0j 

A j = Yio 

Pi j ~ 7 20 ^ r 2l SUN j + U 2 j 

Psj — Y 30 P Yy\ SUN j + C 3 j 


Table 5. Parameter estimates for treatment effects (Model 1) and moderation effects (Model 2) of 
SUN vs. Control in Year 1 

Note. N=770. Standard errors were reported in the parentheses. ** indicates significance at .01 


level and * indicates significance at .05 level. 



DV: Multiple Choice 

DV: Drawing 

DV: Confidence 

Parameters 

Model 1 

Model 2 

Model 1 

Model 2 

Model 1 

Model 2 

Fixed Effects 







Intercept 

6.S5 (0.29)** 

6.88 (0.30)** 

0.93 (0.29)** 

1.06 (0.32)** 

2.81 (0.10)** 

2.78 (0.10)** 

Pre-test 

0.26 (0.05)** 

0.26 (0.05)** 

0.73 (0.51) 

0.78 (0.50) 

0.43 (0.03)** 

0.43 (0.04)** 

GPA 

0.72 (0.14)** 

0.38(0.18)* 

0.75 (0.16)** 

0.29 (0.10)** 

0.13 (0.03)** 

0.07 (0.05) 

Male 

0.64 (0.19)** 

0.52 (0.23)* 

0.15 (0.14) 

0.04 (0.11) 

0.11 (0.05)* 

0.13 (0.06)* 

Days Elapsed 

-0.007 (0.003)** 

-0.006 (0.003)* 

-0.006 (0.003) 

-0.010 (0.003)** 

-0.003 (0.001)** 

-0.003 (0.001)** 

SUN 

1.31 (0.23)** 

1.21 (0.30)** 

1.19(0.44)** 

1.81 (0.37)** 

0.37 (0.08)** 

0.38 (0.10)** 

GPA*SUN 

- 

0.61 (0.26)* 

- 

0.86 (0.26)** 

- 

0.09 (0.07) 
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Male*SUN 

- 

0.19(0.37) 

- 

0.15(0.26) 

- 

-0.04(0.10) 

Random Effects 







Intercept 

0.20 (0.12) 

0.22 (0.12) 

1.14(0.36)** 

0.91 (0.28)** 

0.03 (0.01)* 

0.03 (0.01)* 

GPA 

-- 

0.07(0.12) 

0.55 (0.44) 

0.43 (0.25) 

-- 

0.00 (0.04) 

MALE 

« 

0.03(0.16) 

- 

0.02 (0.24) 

- 

0.00 (0.05) 

GPA with Intercept 

- 

- 

0.66 (0.33)* 

- 



Residual 

5.63 (0.33)** 

5.51(0.33) ** 

2.91 (0.57)** 

2.89 (0.55)** 

0.35 (0.02)** 

0.35 (0.02)** 


Table 6. Parameter estimates for the comparison of SUN and Control vs. AP in Year 1 
Note. N=885. Standard errors were reported in the parentheses. ** indicates significance at .01 
level and * indicates significance at .05 level. 



DV: Multiple Choice 

DV: Drawing 

DV: Confidence 

Fixed Effects 




Intercept 

8.27 (0.66)** 

1.30 (0.48)** 

2.87 (0.02)** 

Pre-test 

0.30 (0.05)** 

1.05 (0.30)** 

0.45 (0.03)** 

GPA 

0.79 (0.15)** 

0.84 (0.18)** 

0.14(0.03)** 

Male 

0.76 (0.19)** 

0.18 (0.14) 

0.16(0.05)** 

Days Elapsed 

-0.004 (0.003) 

-0.005 (0.003) 

-0.002 (0.001) 

SUN 

-0.40 (0.64) 

1.01 (0.52)* 

0.18(0.21) 

Control 

-1.47 (0.62)* 

-0.33 (0.42) 

-0.20 (020) 

Random Effects 




Intercept 

0.56 (0.23)* 

1.18 (0.34)** 

0.07 (0.02) ** 

GPA 

0.38 (0.27) 

0.72 (0.57) 

- 

GPA with Intercept 

0.38(0.18)* 

0.73 (0.30)* 

- 

Residual 

5.53 (0.30)** 

3.06 (0.51)** 

0.35 (0.02)** 
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il. Draw what occurs in the mitochondrion during CELLULAR RESPIRATION 
Show what happens because of moving electrons. Label your drawing Explain 

your drawing in the box below. 

MAS85.20 

Page | IS 



Figure 1. A. Above is the “post” drawing of a regular biology male student in the 
Treatment Group with a 3.9 previous GPA, 117 days after instruction. This student 
exhibits mechanistic thinking in terms of the three interacting systems that allow for 
energy transfer in the mitochondrion: 1) the movement of electrons which causes 2) 
the pumping of protons, which in turn powers 3) the production of ATP by the ATP 
synthase. However, the role of reactants (food + oxygen, which are the electron 
donor and acceptor) and the products made when electrons move (carbon dioxide + 
water) are not referenced. Notice that he is “extremely” confident regarding this 
response. 
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Post SUN vs. Control 



■ SUN ■ Control 


Figure 2. Comparison of percentage of SUN (blue) and Control (orange) students achieving the 
‘drawing with written explanation’ criteria on the post assessment >2.5 months after instruction. 
No adjustments were made for control of variables. N = 388 SUN and 361 Control students. 
Significant differences (p=.00 or.01) between SUN and Control are shown with asterisks (blue 
where SUN is greater). This assessment was developed to include concepts from: 1. General 
lower level criteria regarding structures and their spatial relationships (items 1 - 5); 2. The 
process of cellular respiration (items 6-13); 4. The production of ATP (items 14 - 17), and 
4.Energy transformations (items 18 and 19). 
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Post SUN vs. AP 
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Figure 3. Comparison of percentage of SUN (blue) and AP (purple) students achieving drawing 
with explanation criteria >2.5 months after instruction. No adjustments were made for control of 
variables. N=388 SUN and 106 AP students. Significant differences (p=.00 or .01) between 
SUN and AP are shown with asterisks. Those that favor SUN are blue and those that favor the 
AP group are purple. This assessment was developed by the project to include concepts from 
four broad categories: 1. General lower level criteria regarding structures and their spatial 
relationships (items 1 - 5); 2.The process of cellular respiration (items 6-13); 4. The production 
of ATP (items 14 - 17), and 4.Energy transformations (items 18 and 19). 
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